The development of a general purpose controller capable of analysing visual information derived from one or more cameras (or other forms of electronic imaging devices), and using the results of the analysis to devise strategies for accomplishing manipulative tasks in real time, has attracted great interest in robotic circles in recent years. This is because a robot without some such external sensory input moves within its workspace without knowledge of its external environment, and derives orientation information only from the joint resolvers which form part of its servo control mechanism. This imposes severe limitations on the tasks that the robot can undertake and can make the costs associated with the installation very high.
Potential applications of an adaptive vision-based control system include:
Identification of objects PA1 Location of objects and determination of their orientation PA1 Inspection and Quality Control PA1 Visual servoing PA1 Navigation and scene analysis PA1 Complex inspection PA1 Automated assembly tasks PA1 1. A `Knowledge Base`, which defines the domain of operation and contains data models of plausible objects; and PA1 2. A `Task Description`, which defines the task. PA1 1. `Low-Level` subsystems which are concerned with scene analysis and the extraction of features from images, using standardised procedures to extract hierarchical descriptions of scenes from camera images; PA1 2. `High-Level` subsystems, which recognise objects in the scene by comparing the hierarchical descriptions with the knowledge base to produce a highly refined description of the scene called the `World Model`; and PA1 3. `Intermediate` subsystems, which combine elements of 1 and 2. PA1 1. A `Segmenter`, which decomposes the image into regions of approximately uniform hue (or, optionally, colour saturation or image intensity); PA1 2. A `Sketcher`, which generates a `Sketch` of the scene composed of arcs each of which represents a significant feature; PA1 3. A `Flow Subsystem`, which produces pixel-velocity maps by comparing pairs of slightly different images; PA1 4. A `Texture Subsystem`, which analyses the image in terms of texture; PA1 5. A `Tracking Subsystem`, which tracks fiducial regions and features. PA1 1. `Rangers`, which generate depth and scale information from multiple views (derived either from a plurality of cameras or from several images from the same camera); and PA1 2. `Controllers`, which drive effectors associated with the robot arm in order to achieve a desired action as directed by the planner. PA1 1. The `Recogniser`, a committee of high-level subsystems which together use hierarchical scenic information and feature maps produced by the low- and intermediate-level subsystems in conjunction with the knowledge base go construct a world model, that is to say a logical representation of the working envelope of the robot. PA1 2. The `Planner`, a group of high-level subsystems which uses the world model and the task description to construct a plan which is a proposed set of actions for accomplishing the task. The planner also generates internal demands for further information from other subsystems as and when it is required. PA1 the adaptation and integration of hitherto separate image analysis techniques; PA1 coupling these techniques with a decision-making function so as to provide an adaptive control system for the control, for example, of a robot arm; PA1 substantially reducing the amount of data present in an image so that only the most important information remains; PA1 producing from raw image data several progressively defocused images ranging from a finely defocused image to a coarsely defocused image; PA1 detecting closed contours in defocused images and comparing contours thus detected in successive pairs of defocused images; PA1 producing a map of partial contours in the most finely defocused image which correspond to parts of contours in the other progressively defocused images; PA1 using partial contours to identify important features of a scene; PA1 partitioning an image into regions of similar hue, intensity or saturation; PA1 analysing an image in terms of its texture at each point of the image; PA1 analysing an image in terms of its apparent pixel velocity at each point; PA1 analysing an image in terms of vertices and the connecting relationship between these vertices; PA1 defining image colour data as a unique scalar quantity by the use of mathematical functions with non-integer dimensional characteristics; PA1 automatically determining the characteristics of a camera by comparing the image of an object with an internally stored model of the object; PA1 obtaining depth information from two or more cameras by triangulation; PA1 obtaining depth information from a single camera by comparing images of a scene separated in time; PA1 tracking image data in a manner which provides a solution to the so-called correspondence problem; PA1 tracking image data in order to find range or to assist in more accurate range finding; PA1 using the tracking techniques as a tool to assist searching and correlation of image data; PA1 comparing image information with a suite of internally stored representations of known objects so as to identify objects and their locations in three-dimensions; PA1 making reasoned task-oriented decisions based on the extracted information; PA1 decomposition decisions into primitive action commands which actuate servo motors which drive a robot arm, or some other physical device, through its workspace; and PA1 embedding the early vision functions in hardware thereby enabling real-time processing.
Hitherto, such vision-based controllers as have existed have been limited, special-purpose devices developed to perform specific industrial tasks. Thus any gain in speed has been achieved only by using dedicated vision systems architecture for specialist operations such as for example locating a frame on a car body during the insertion of a windscreen and these dedicated systems have required modifications in both the hardware and the software in order for the system to perform other tasks within the same generic classification. So far as the inventors are aware, there has hitherto been no successful utilisation of a truly general purpose vision-based controller.